Scaling Relationships in Back-propagation Learning

A bstrac t. We present an empirical st udy of th e required training time for neural networks to learn to compute the parity function using the back -propagation learning algorithm, as a function of t he numb er of inp uts. The parity funct ion is a Boolean predica te whose order is equal to th e number of inpu t s. \Ve find t hat t he t rain ing time behaves roughly as 4" I where n is the num ber of inp ut s, for values of n between 2 and 8. T his is consistent with recent t heoretical analyses of similar algorit hms. As a part of thi s stu dy we sea rched for optimal par ameter tunings for each value of n. We suggest that the learning rate should decrease faster than lin, the momentum coefficient should approach 1 exponentially, and the initial random weight scale should remain approximately constant.